|
|
Algonauts challenge 2023: predicting human fMRI activity in response to visual stimulation
Petliak, Nataliia ; Antolík, Ján (advisor) ; Bojar, Ondřej (referee)
In this thesis, we investigate the application of pretrained Deep Neural Networks, par- ticularly Vision Transformers (ViT), for predicting human fMRI activity in response to visual stimulation. The Algonauts Challenge 2023 dataset, serving as a large-scale bench- mark of human fMRI data, allows us to assess the performance of ViT in comparison with established CNN architectures like VGG and ResNet. Our study highlights the complex- ity of this task, especially in accurately modeling the diverse regions of the full visual cortex. We identify specific ViT layers that align with the brain's hierarchical processing and prove to be the most predictive. However, one of the limitations we encounter with pretrained ViT is its reduced adaptability due to inherent subject variability. This limi- tation underscores the challenge in developing a single model that is universally effective across different individuals. To address this, we implement an iterative training strategy, starting with the layers that perform best across all subjects, followed by fine-tuning for specific visual areas in individual subjects. Despite these efforts, the effectiveness of ViT varies; it performs satisfactorily in some subjects but struggles in others, particu- larly in word-selective regions. The incorporation of textual data...
|
guest ::
